Apr 11, 2005 2:37am
Results from Deriver Evaluation
Here's a summary of the feedback we got about the idea of having supplemental lossy files alongside the archival lossless material here. After a year of testing, we held an evaluation period starting in February, see http://www.archive.org/iathreads/post-view.php?id=28774
We asked for reports of both positive and negative impacts- whether here in the archive, for bands, in people's lives, or in their communities (for example, trading communities). A wide variety of people responded: regular downloaders, uploaders, tapers, artists, staff and volunteers, even a reported non-user or two. There were ~50 active participants in the discussion.
We received many reports of significant positive impact. The main reasons for happiness with supplemental lossies were as follows:
-Ability to stream or preview to be exposed to unfamiliar bands (the most-reported reason by far!)
-Ability to stream or preview before a planned download in lossless form
-Ease of use (streaming at work, portability to car or portable listening device, familiarity of formats)
-Ease of access for low bandwidth (dialup) or limited storage capacity
-Increased exposure to the LMA in general, as a collection
The Archive itself does not appear to have been negatively affected. The level of submissions to the LMA has not decreased (data analyzed were outside periods of large special-project uploads). One proposed yardstick, membership at IA, was not useful since no membership is required for LMA participation, and many other non-LMA collections may influence membership.
We did not hear from many artists themselves within the discussion thread. However, over the last year through email, curators have received dozens of endorsements of the supplemental idea from various OK'd artists here.
Some folks still had misgivings about negative impact on principle, in the same vein as the misgivings when the idea was introduced a year ago. Yet despite a lot of calls for reports, no one was able to demonstrate actual negative impact on the archive or on their communities. An example of two tapers known to withhold their material from the LMA was shown not to be relevant, since they do so for philosophical reasons predating and beyond the deriver idea.
Still others (users, tapers and artists) reported that their misgivings about lossies had fallen away over the last year. The primary reasons for comfort appear to be that the deriver system remains optional, and that a lossless fileset remains accessible as the primary copy. Many people also reported satisfaction with the Archive's serving as a quality control point for lossies that do come into being: there is less worry about possibly inferior deriviations being made by others. Feelings all around were strong that the deriver system should remain optional in the same way that it is now (with accomodations for bands', tapers' or uploaders' wishes), and that lossless formats should continue to be available.
However, there were a number of suggestions for ways to improve the deriver system further. Among these:
-Better attention to deriver bugs and/or better communication about them when they occur
-Ogg streaming should be added
-FLAC streaming could be added?
-Label streams more clearly (as high vs. low-bandwidth choices)
-A few calls for streaming-only lossy, but not downloadable (staff reports this is not doable)
-Requirement for FLAC tagging before or after upload
-Add a categorization system for content (keywords for artists? "what's related?") so users won't have to rely as heavily on streaming for exposure